The Claims: 

This listing of claims will replace all prior versions, and listing of claims in the 
application. 

1 . (Original) A method for integrated retrieval and annotation of 
stored images involving a computer application in which text is input by a user, 
said method comprising the steps of: 

a) providing metadata associated with the stored images; 

b) running a user application in which text is entered by a user; 

c) continuously monitoring text typed by the user into the 
application to isolate the context expressed by the text; 

d) matching the context with the metadata associated with the 
stored images, thereby providing one or more matched images; 

e) retrieving and displaying the matched images; and 

f) utilizing the context to provide suggested annotations to the user 
for the matched images, together with the capability of selecting certain of the 
suggested annotations for subsequent association with the matched images. 

2. (Original) The method as claimed in claim 1 further 
comprising the step of providing the user with the capability of inserting selected 
ones of the matched images into the text of the application. 

3. (Original) The method as claimed in claim 1 wherein the 
stored images are stored in a data base and step f) further includes automatically 
updating the metadata for the matched image stored in the data base with the 
suggested annotations. 

4. (Original) The method as claimed in claim 1 wherein the 
metadata includes keywords related to the images and the context likewise 
comprises keywords. 

. 5. (Original) The method as claimed in claim 1 wherein the 

metadata includes content data related to the images and the context likewise 
comprises content data. 



6. (Original) The method as claimed in claim 1 wherein the 
metadata includes temporal data related to the images and the context likewise 
comprises temporal data. 

7. (Original) The method as claimed in claim 1 wherein the user 
is entering text at a particular cursor location and the context is determined based 
upon the proximity of typed words to the cursor location. 

8. (Original) The method as claimed in claim 1 further 
comprising the step of providing the user with the capability of adding other 
annotations to the matched images. 

9. (Original) The method as claimed in claim 1 wherein step d) 
further comprises ranking the matched images according to context, thereby 
providing ranked images that may be displayed in step e) according to their rank. 

10. (Original) The method as claimed in claim 9 wherein the 
context comprises keywords and the ranking is based on the number of matched 
keywords for each matched image. 

1 1 . (Original) The method as claimed in claim 9 wherein the 
context and metadata include temporal data and the ranking is based on the 
temporal distance between the context and the metadata associated with each 
matched image. 

12. (Original) The method as claimed in claim 9 wherein the 
context comprises keywords and temporal data and the ranking is based on the 
intersection of their occurrence for each matched image. 

13. (Original) A method for utilizing images that are stored in a 
data base with an e-mail application in which text is input by a user into a 
message, said method comprising the steps of: 

a) providing metadata associated with the stored images; 



b) running an e-mail application in which text is entered by a user 
into a message window; 

c) continuously monitoring text typed by the user into the message 
window to isolate keywords in the text; 

d) matching and ranking the keywords with the metadata 
associated with the stored images, thereby providing one or more matched images 
that are ranked according to the keywords; 

e) retrieving and displaying the matched images alongside the 
message window in order of their rank; and 

f) providing the user with the capability of moving selected ones of 
the matched images into the message window. 

14. (Original) The method as claimed in claim 13 further 
comprising the step of utilizing the context to automatically provide new keyword 
annotations for the matched images. 



15. (Original) The method as claimed in claim 13 further 
comprising the step of utilizing the context to provide new keyword annotations 
for the matched images, together with the capability of manually selecting certain 
of the annotations for subsequent association with the matched images in the data 
base. 



16. (Original) The method as claimed in claim 14 wherein the 
step of utilizing the context further includes automatically updating the metadata 
for the matched image with the annotations. 

17. (Original) The method as claimed in claim 15 wherein the 
step of utilizing the context further includes automatically updating the metadata 
for the matched image with the annotations. 

1 8. (Original) The method as claimed in claim 1 3 wherein the 
ranking is based on the number of matched keywords for each matched image. 



19. (Original) The method as claimed in claim 13 wherein the 
keywords and metadata include temporal references and the ranking is based on 
the temporal distance between the keyword and the metadata associated with each 
matched image. 

20. (Original) The method as claimed in claim 1 3 further 
comprising the step of utilizing the context to either automatically or manually 
select new keyword annotations for the matched images, wherein the ranking is 
based on a higher confidence level for the manually selected keyword 
annotations. 

2 1 . (Original) The method as claimed in claim 1 3 wherein the 
keywords and metadata include spatial or geographic references and the ranking 
is based on the spatial or geographic correspondence between the keyword and 
the metadata associated with each matched image. 

22. (Original) An autonomous agent for use in automatically 
interfacing a text-based application with a picture archive containing 
representations of images and metadata associated with the images, said agent 
comprising: 

a language analyzer for continuously monitoring text entered by 
the user into the application and extracting keywords appropriate to the context 
surrounding the entered text; 

a retrieval agent for retrieving images from the archive on the basis 
of one or more matches between the extracted keywords and the metadata; 

an annotation agent for providing suggested annotations to the user 
for the matched images based on the extracted keywords, together with the 
capability of selecting certain of the suggested annotations for subsequent 
association with the matched images; and 

a picture database viewer for displaying the retrieved images to the 
user and for enabling the images to be incorporated into the application, as desired 
by the user. 



23. (Original) The agent as claimed in claim 22 wherein the 
retrieval agent ranks the retrieved images based on attributes of the keywords, and 
the picture database viewer displays the retrieved images in order of their ranking. 

24. (Original) The agent as claimed in claim 22 wherein the 
application is an e-mail application. 

25. (Original) The agent as claimed in claim 22 wherein the 
picture database viewer incorporates a retrieved image into the application by a 
drag and drop operation. 

26. (Original) The agent as claimed in claim 22 wherein the 
picture database viewer incorporates a retrieved image into the application by a 
one click selection of the image. 

27. (Original) The agent as claimed in claim 22 further 
comprising a manual annotation agent for adding annotations to the retrieved 
image and for storing the added annotations to the metadata stored in the archive. 

28. (Original) The agent as claimed in claim 22 wherein the 
retrieval agent retrieves images from the archive on the basis of matching between 
the extracted keywords and temporal metadata identifying the time and date when 
the images were captured. 

29. (Original) The agent as claimed in claim 22 further 
comprising an automatic import agent for automatically entering images into the 
archive from one or more specified image sources. 

30. (Original) The agent as claimed in claim 29 wherein the 
retrieval agent automatically retrieves the entered images and the picture database 
viewer automatically displays the entered images. 



